

Backpropagation Neural Network Architecture 

The BP network implements the generalized delta rule. It is a gradient 
descent algorithm which minimizes the squared error of the network. The 
gradient descent algorithm is applied to adjust the connected weights. 

The training process of the BP neural network generally involves five steps: 

1. Select representative training samples and turn them into the input 
layer as the input value. 

2. Calculate the predictive value of the network. 

3. Compare the target value with the predictive value to obtain the error 
value. 

4. Readjust the weights in each layer of the network according to the 
error value. 

Repeat the above procedure until the error value of each 
training sample is minimized, meaning that the training is 
finished . 
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The following is the scenario: 

The output of the kth output neuron, s k is: 

Sk = fiVk) 

where y k is the network input to the kth output neuron which is written as: 

yic = ^ f(.Zj ) w jk 
j 

where / (z 7 ) is the output of the jth hidden unit 

where Zj is the network input to the jth hidden neuron which is written as: 

z j = ^ XiWij 
i 

where x t is ith input unit and is the weights of the hidden layer. 




The error function to be minimized is: 


n 

k = 1 

where d is the desired output vector. 



for weight connected from hidden to output units: 


d£ _ d£ d/(y fc ) 
dw jk df(y k ) dy k dw jk 


where w jk is the weight of the output layer. 

let fiyk) = sigmoidal function 


dE 

dw jk 


~(d - /(y fc ))/(y fc )( l - /(y fc )) ^ 


dE 

dw Jk 


e f(yk){ 1 - /(yfc)) f(Zj ) 


Let S k = e /(y k )(l - /(y fc )) 

dE 
dw jk 


= ~ S k f(z,) 


for weight connected from input to hidden units: 


= _(sr dff df (y k ) dy k \ df (z, ) dz ; 
dwij \Z-,df(_y k ) dy k df(z j )J dzj dw t] 
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- ~ -e /(y fc )(l -/(yfc)) Xi 


dE 

dw ti 


Sj 


- S k W Jk ^ f(Zj )(1 - f(Zj )) Xi 

S k Wj k ^ /(Zy)( 1 - /(Zy )) 
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Let 


the change in weights of the output units are given by 


Aw,* = -77 


dE 


dw / 


Ok 


= rjS k f (zj ) 

where 77 learning rate parameter. 

The change in weights of the hidden units is 

dE 

Aw U = 


= V Sj x, 


The weights update equations are given in 

Wj k new = w jk old +A w Jk 
Wtj new = Wij old + A w tJ 


where Wj t new 


and Wji new are the new weights, Wj t old and Wji old are the previous weights. 



